AUS920010469US1 
Craddock et o\. 
PCI Migration Semantic Storage I/O 



1/11 



iz55LjQ 



ti'i 




AUS920010469US1 
Craddock et al. 
PCI Migration Semantic Storage I/O 

■2/1 1 



host processor node 

(consumer) (consumer) (consumer) ° ° ° (consumer) 
v -7^ ^j^s ^ 

202 204 206 208 
224 s ( MESSAGE AND DATA SERVICE ) 222 
VERBS 



CHANNEL ADAPTER (ENDN0DE) 





PORT 


o o o 


PORT 




/ 



"7 210 ( r 

214 _ 216 



CHANNEL ADAPTER (ENDNODE) 



PORT 


o o o 


PORT 


s 

212 


S 

218 


S 

220 



FIG. 2 




302A 304A 306A 308A 



TRANSPORT 



VL 



VL 



7 T 

31 8A 320A 
PORT 

~~r~ 

31 2A 



VL 



322A 



VL 



VL 



7 T 
324A 326A 

PORT 
314A 



VL 



328A 



VL 



VL 



7 T 

330A 332A 
PORT 



FIG. 3A 



31 6A 



VL 

T 

334A 



AUS920Q10469US1 
Craddock et al. 
PCI Migration Semantic Storage I/O 

3/11 



300B 



SWITCH 



302B 

_z_ 



PACKET RELAY 



VL 



VL 



VL 



VL 



VL 



VL 



VL 



VL 




VL 


ooo 





-306B 



PORT 



PORT 



PORT 



-304B 



FIG. 3B 



300C 



ROUTER 



302C 



GRH PACKET RELAY 



VL 



VL 



VL 



VL 



VL 



VL 



VL 



VL 




VL 


OOO 





-306C 



PORT 



PORT 



PORT 



-304C 



FIG. 3C 



AUS920010469US1 

Craddock et al. 
PCI Migration Semantic Storage I/O 



FIG. 4 

406 

\ 



CONSUMER 



Mm 



WORK 
REQUEST 

~r~ 

410 
412 

WORK 
REQUEST 



> 



> 



414 



WORK 
COMPLETION 



4/11 



DATA 
SEGMENT 



DATA 
SEGMENT 2 



■444 



■446 



DATA 
SEGMENT 3 



■448 



400 



, | RECEIVE WORK QUEUE 



HWQE l 
— i 



416 



WQE 



T 
418 



WQE 



T 
419 



WQE 

7~ 



420 



DATA 
SEGMENT 4 



I 



DATA 
SEGMENT 5 



DATA 
SEGMENT 6 



■438 



■440 



■442 



SEND WORK QUEUE 



h"wqe"I 

422 424 



WQE 

7~ 



WQE 



426 



l i 

l l i— 

J L_ 



WQE 



T 
428 



■402 



COMPLETION QUEUE 



CQE 
430 



CQE 



T 
432 



CQE 



7 
434 



I r 

I CQE h 
436 




-404 



ACTIVE SIDE PASSIVE SIDE 

REQ (PRIVATE DATA) 
^ REP OR REJ (PRIVATE DATA) 
RTU OR REJ (PRIVATE DATA) _ 



FIG. 1 2 



AUS920010469US1 
Craddock et al. 
PCI Migration Semantic Storage I/O 



FIG. 5 

500 



510- 



5/1 1 



PROCESSOR 1 



PROCESS A 
QUEUE PAIR 4 



SEND 



RECEIVE 



QUEUE PAIR 7 



SEND 



RECEIVE 



QUEUE PAIR 6 



SEND 



RECEIVE 




PROCESSOR 2 




PROCESS C 
QUEUE PAIR 2 



SEND 



RECEIVE 



PROCESS D 
QUEUE PAIR 5 



SEND 



RECEIVE 



PROCESSOR 3 




PROCESS E 
QUEUE PAIR 9 



SEND 



RECEIVE 



610 



FIG. 6 

600 



PROCESSOR 1 



PROCESS A 
QUEUE PAIR 4 



SEND 



RECEIVE 



HCA DLID=33 



PROCESSOR 2 




PROCESS C 
QUEUE PAIR 24 



SEND 



RECEIVE 



PROCESS D 
QUEUE PAIR 25 



SEND 



RECEIVE 



HCA DLID=27 



PROCESSOR 3 



PROCESS E 
QUEUE PAIR 14 



SEND 



RECEIVE 



HCA DLID=54 



AUS920010469US1 
Craddock et al. 
PCI Migration Semantic Storage I/O 

6/1 1 



FIG. 7 

70(K 

708 ^ 
716 718 


MESSAGE DATA 

DATA DATA DATA 
SEGMENT 1 SEGMENT 2 SEGMENT 3 
702 704 706 




PACKET 




i 710 
! / 


714 




ROUTING 
HEADER 


TRANSPORT 
HEADER 


PACKET PAYLOAD 
j — 


CRC 

/ 



712 

DATA PACKET (ROUTED UNIT OF WORK) 



802 



816- 



800 

HOST PROCESSOR NODE 



FIG. 8 



804 

_A_ 



PROCESS A 
QP 23 



824 



806- 









" SEND 




RECEIVE 


> 


V, 


4 



-826 



820 
HCA 23 



PORT 



V T 



PORT 
-7— 



HOST PROCESSOR NODE 



818- 



PROCESS B 
QP 24 



828- 



808- 









■ SEND 




RECEIVE " 


N 


— S 


* 



■830 



822 
HCA 24 



PORT 



A 



^ T 



PORT 



810 



A 



SWITCH 



812 



814- 



SWITCH 



AUS9200 10469 US 1 
Craddock et aL 
PCI Migration Semantic Storage I/O 



7/11 



QZ 



CO 



<zd 
o 

LO 



C£ 


I — 




LU 


CO 
LU 


CO 




CO 
LU 






ADA 


o o 

LU O 
QZ OZ 


I 

LU 


o 


Q_ 
i 


MNVH 


CO 
LU 


_S0H 


o 


> 




i — 


LU 
O 


o 


CO 


ct: 


o 


LU 

crz 


u_ 


□z 





o 

LO 

A 



o 



CO 
LU 

tc 
o 
o 



CO 

o 



CO 

o 

CO 
CO 

ct: 

LU 
Q_ 

CO 



o 

LO 

1 



CO 
LU 

o 

LU 

az 

CO 



ct: 



CD 



CO 

o 



CO 
LO 



CO 



ZD OO 

old 

LU t 

^ s 

CO C£ 
LU |— 

oz 

lu >- 

Z Od 
LU O 



CO 

o 



00 

o 

LO 



CO CO 



Lu QQ 
co |= 

O 
I— ^ 

CO LU 

o 



CD 
LO 

to 

LU 
ZD 

a 

LU 

I 

ct: 
ct: 



CD 



CN| 
LO 



a: 



CO 



o 

OZ 



LO 



CO 

o 

CL. 
CO 
LU 

oz 

CO 



CO 

ct: 



CL. 

o 



"= — \ ct: co 

CD VLU CD 



CNJ 

CD Vq_ 



Q_ 

CD O 

ZD OZ 
LU O 

LU <C 

^ o 



CO 


O 






s 


I 




i — 

C£ 


LU 


CNl 




o 




LU 

a: 


Q_ 


az 


o 




o 


h— 


o 


PER 


^ 


Q_ 




ct: 


ZD 


oz 




o 

LU 




o 

LU 


ID'S 




O 




ZD 


o 


CL. 


o 


CD 







Q_ 
O 



CD 
O 



CL. 
CT 



o 
or 



-t\ /r 

:IVE 




o 




LU 

OZ 




1 1 


SEND 


1 
1 




:IVE 




o 

LU 




ct: 






CD 

~zz 

LU 




CO 




: i 


ICEIVE 




t i — i 

OZ 




1 1 


:nd 




CO 





3 



ct: 
o 

-< 

o 








1002 



AUS920010469US1 
Craddock et al. 
PCI Migration Semantic Storage I/O 

8/11 

LIDs ARE UNIQUE ONLY 
WITHIN A SUBNET 



1000 




ENDNODE 



-1006 











ENDNODE 



-1008 



PORT 
PORT PORT 




ENDNODE 



■1010 



PORT 
PORT PORT 



PORT 



SWITCH 
1 



PORT 




PORT 
PORT PORT 




ROUTERS 
PROVIDE 
CONNECTIVITY 
AMONG 
SUBNETS 



1024- 

ALL GIDs WITHIN 
A SUBNET SHARE 

THE SAME 
SUBNET PREFIX 



PORT 



ROUTER 
3 



PORT 




PORT PORT 
PORT 



1 026- 



PORT 



ROUTER 
4 



PORT 




PORT 
PORT PORT 




PORT PORT 
PORT 



PORT 
PORT PORT 



SWITCH 
5 



PORT 



PORT . PORT 
PORT 




PORT 



SWITCH 
6 



PORT 




PORT PORT 
PORT 



ENDNODE 



FIG. 10 



AUS92001 0469-US1 

Craddock et al. 
PCI Migration Semantic Storage I/O 



9/1 1 



LO 



/ 
I 
I 

N 



CO 



o 
o 



o 
o 



CO 

o 

LU 
Q- 

_o 
en 



CO 

o 
o 



Cn 



I o I 



I 



o 



o 
o 
o 



CM ^ 

^" LU O 
— Q_ cn 

a- a_ 



00 
o 

— en 

LU 

o 



cn 

CO 



cn 
o 



o 
o 



CO 
CO 

LU _j 
O o 
O cn 
<£ H- 

^ O 
O O 



o 

CO 
LU 

% 

CO 
CO 



CO 



cn 



o 

Cn 

CQ 
ZD 
CO 

Cn 



5 

LU 

ct: 
o 



o 



CO 



ct: 
1 

o 
o 

Cn 

t] 

CD 
3) 
CO 



5 

LU 


MAC 


a: 








PACK 


MAC 



> or 1 



00 
o 

CD <C 

— . cr: 

LU 

o_ 
o 



en 

CO 



in 
en 
o 



3 
o 
o 



1 



CO 
CO 

LU __j 
O o 

o cn 

<C 1— 

:£ o 
o o 



^— 






QC 






0 


ct: 

LU 




Q_ 


1 CO 


CO 




1 0 




5 




£ 







in 

cn cn 

O LU 



cn 

LU 



CO 



O 



CO 
.CNJ 



ct: 


LO 


LU 




h— 

ZD 




O 




cn 





J 



O 
CO 



ro 



o 

CSS 



CZl 

o 



[CAI 


on 

LU 






□Z 




Q_ 









AUS920010469US1 
Craddock et al. 
PCI Migration Semantic Storage I/O 



10/11 




o 



AUS92O01O469US1 
Craddock et al. 
PCI Migration Semantic Storage I/O 

11/11 



LU Q 
Or 5 



1 . 1 




CQE 


RDM 



LU 


LU 
>; 


Or 


LU 


O 


O 
LU 







a. yj 



oo 



go 

Qi 



CO 



5h 



CO 

ro 











LU 
> 








LU LU 


LU 






O 




as 

Qi ^ 


LU 
Qi 











o 

ro 



3£ 

2 co 



CNI 



CN CnJ 



Qi 



Qi 



Qi 



o 

Qi 



LU 
Qi 



Qi 



DO 

ro 



LU 
LU j< 

2 S 



Qi I — 



o 

LU 

CO 



7] /\ J 




